Method and terminal for reproducing content

ABSTRACT

A method of reproducing content is provided. The method performed by a terminal includes operations of obtaining metadata corresponding to a predetermined portion of the content, selecting, based on the metadata, at least one external device to reproduce the predetermined portion of the content, transmitting a control command to the at least one external device, and reproducing the predetermined portion of the content by interoperation of the terminal with the at least one external device to which the control command is transmitted.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation application of prior application Ser.No. 14/934,820, filed on Nov. 6, 2015, which claimed the benefit ofprior application Ser. No. 14/132,907, filed on Dec. 18, 2013, which hasissued as U.S. Pat. No. 9,210,367 on Dec. 8, 2015 and claimed thebenefit under 35 U.S.C. § 119(a) of a Korean patent application filed onDec. 21, 2012, in the Korean Intellectual Property Office and assignedSerial number 10-2012-0150931, and of a Korean patent application filedon Mar. 8, 2013, in the Korean Intellectual Property Office and assignedSerial number 10-2013-0025255, the entire disclosure of each of which ishereby incorporated by reference.

TECHNICAL FIELD

The present disclosure relates to a method and terminal for reproducingcontent along with an external device connected to the terminal.

BACKGROUND

As a result of on-going developments in communication technology, anapplication may be simultaneously executed on a device as well asvarious peripheral devices connected to the device. In addition, thedevice may control the various peripheral devices via other application.

In addition, as various peripheral devices that can be connected toother device via an application are nowadays widely manufactured, thereis a demand for a content reproduction system that allows a user toreproduce content in various ways by using various peripheral deviceswhen the user watches the content via another device.

The above information is presented as background information only toassist with an understanding of the present disclosure. No determinationhas been made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the present disclosure.

SUMMARY

Aspects of the present disclosure are to address at least theabove-mentioned problems and/or disadvantages and to provide at leastthe advantages described below. Accordingly, an aspect of the presentdisclosure is to provide a method of reproducing, by a terminal, apredetermined portion of content by interoperation of the terminal andan external device connected thereto.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the present disclosure, a method ofreproducing content is provided. The method includes operations ofobtaining metadata corresponding to a predetermined portion of thecontent, selecting, based on the metadata, at least one external deviceto reproduce the predetermined portion of the content, transmitting acontrol command to the at least one external device, and reproducing thepredetermined portion of the content by interoperation of the terminalwith the at least one external device to which the control command istransmitted.

In accordance with another aspect of the present disclosure, the methodmay further include operations of executing a first application forreproducing the content and executing a second application forcontrolling the at least one external device.

In accordance with another aspect of the present disclosure, the firstapplication may transmit control information included in the metadata tothe second application, and the second application may convert thecontrol information into the control command according to a controlprotocol of the at least one external device.

In accordance with another aspect of the present disclosure, the contentmay include at least one of education content, movie content,broadcasting content, and game content.

In accordance with another aspect of the present disclosure, themetadata may include at least one of information about the predeterminedportion, IDentification (ID) information of the at least one externaldevice, and capability information about a capability to be performed bythe at least one external device.

In accordance with another aspect of the present disclosure, theoperation of obtaining the metadata may include an operation ofextracting the metadata inserted into the content.

In accordance with another aspect of the present disclosure, theoperation of obtaining the metadata may include an operation ofreceiving from an external source and separately from the content themetadata corresponding to the predetermined portion of the content.

In accordance with another aspect of the present disclosure, theoperation of obtaining the metadata may include an operation ofgenerating the metadata corresponding to the predetermined portion ofthe content by performing image processing on the content.

In accordance with another aspect of the present disclosure, theoperation of generating of the metadata may include an operation ofgenerating the metadata corresponding to the predetermined portion ofthe content by considering a type of the at least one external deviceconnected to the terminal.

In accordance with another aspect of the present disclosure, theoperation of generating of the metadata may include operations ofdetecting connection of another external device while the content isreproduced and generating the metadata corresponding to thepredetermined portion by further considering the connection of the otherexternal device.

In accordance with another aspect of the present disclosure, whenreproduction of the content is completed, the method may further includean operation of deleting the metadata.

In accordance with another aspect of the present disclosure, when thecontent is deleted, the method may further include an operation ofdeleting the metadata.

In accordance with another aspect of the present disclosure, theoperation of obtaining the metadata may include operations oftransmitting the content to a gateway that connects the terminal and theat least one external device, and contemporaneously requesting themetadata corresponding to the predetermined portion of the content, andreceiving the metadata corresponding to the predetermined portion of thecontent from the gateway that has generated the metadata by performingimage processing on the content.

In accordance with another aspect of the present disclosure, the methodmay further include an operation of establishing connection with the atleast one external device before the predetermined portion of thecontent is reproduced.

In accordance with another aspect of the present disclosure, theoperation of establishing the connection with the at least one externaldevice may include an operation of performing short-distancecommunication with the at least one external device, and theshort-distance communication may be performed based on at least one of aWireless Local Area Network (wireless LAN), Bluetooth, Wi-Fi Direct(WFD), Ultra WideBand (UWB), and ZigBee.

In accordance with another aspect of the present disclosure, theoperation of selecting may include an operation of selecting of the atleast one external device the at least one external device that is toreproduce the predetermined portion of the content and that is fromamong a plurality of external devices connected to the terminal.

In accordance with another aspect of the present disclosure, the methodmay further include an operation of displaying a list of the pluralityof external devices connected to the terminal.

In accordance with another aspect of the present disclosure, theoperation of transmitting the control command may include an operationof transmitting the control command to the at least one external devicevia a gateway that connects the terminal and the at least one externaldevice.

In accordance with another aspect of the present disclosure, the gatewaymay convert the control information into the control command accordingto a control protocol of the at least one external device.

In accordance with another aspect of the present disclosure, when thecontrol command is transmitted to the at least one external device, theoperation of reproducing of the predetermined portion of the contentincludes an operation of discontinuing reproduction of the content, andwhen a message corresponding to the control command is received from theat least one external device, the operation of reproducing includes anoperation of resuming reproduction of the discontinued content.

In accordance with another aspect of the present disclosure, the controlcommand may include a command for controlling the at least one externaldevice to display a question included in the content, and the messagecorresponding to the control command may include a response message withrespect to the question included in the content.

In accordance with another aspect of the present disclosure, the controlcommand may include a command for controlling the at least one externaldevice to perform setting according to a setting value included in thecontent, and the message corresponding to the control command mayinclude a completion message with respect to the setting.

In accordance with an aspect of the present disclosure, a terminal isprovided. The terminal includes a memory configured to store at leastone program and a control unit configured to reproduce content byexecuting the at least one program, wherein the at least one programincludes commands for obtaining, by the terminal, metadata correspondingto a predetermined portion of the content, selecting, by the terminal,at least one external device to reproduce the predetermined portion ofthe content, based on the metadata, transmitting, by the terminal, acontrol command to the at least one external device, and reproducing, bythe terminal, the predetermined portion of the content by interoperationof the terminal with the at least one external device to which thecontrol command is transmitted.

In accordance with another aspect of the present disclosure, the controlunit may execute a first application for reproducing the content, and asecond application for controlling the at least one external device.

In accordance with another aspect of the present disclosure, the controlunit may extract the metadata inserted into the content.

In accordance with another aspect of the present disclosure, theterminal may further include a communication unit configured to receivethe metadata corresponding to the predetermined portion of the contentfrom an external source, separately from the content.

In accordance with another aspect of the present disclosure, the controlunit may generate the metadata corresponding to the predeterminedportion of the content by performing image processing on the content.

In accordance with another aspect of the present disclosure, the controlunit may generate the metadata corresponding to the predeterminedportion of the content by considering a type of the at least oneexternal device connected to the terminal.

In accordance with another aspect of the present disclosure, the controlunit may detect connection of another external device while the contentis reproduced, and may generate the metadata corresponding to thepredetermined portion by further considering the connection of the otherexternal device.

In accordance with another aspect of the present disclosure, whenreproduction of the content is completed, the control unit may deletethe metadata.

In accordance with another aspect of the present disclosure, when thecontent is deleted, the control unit may delete the metadata.

In accordance with another aspect of the present disclosure, theterminal may further include a communication unit configured to transmitthe content to a gateway that connects the terminal and the at least oneexternal device and to contemporaneously request the metadatacorresponding to the predetermined portion of the content, and then toreceive the metadata corresponding to the predetermined portion of thecontent from the gateway that has generated the metadata by performingimage processing on the content.

In accordance with another aspect of the present disclosure, the controlunit may establish connection with the at least one external devicebefore the predetermined portion of the content is reproduced.

In accordance with another aspect of the present disclosure, the controlunit may select the at least one external device that is to reproducethe predetermined portion of the content and that is from among aplurality of external devices connected to the terminal.

In accordance with another aspect of the present disclosure, theterminal may further include a display unit configured to display a listof the plurality of external devices connected to the terminal.

In accordance with another aspect of the present disclosure, theterminal may further include a communication unit configured to transmitthe control command to the at least one external device via a gatewaythat connects the terminal and the at least one external device.

In accordance with another aspect of the present disclosure, when thecontrol command is transmitted to the at least one external device, thecontrol unit may discontinue reproduction of the content, and when amessage corresponding to the control command is received from the atleast one external device, the control unit may resume reproduction ofthe discontinued content.

In accordance with another aspect of the present disclosure, a systemfor reproducing content is provided. The system includes at least oneexternal device, and a terminal that is operatively connected to the atleast one external device. The terminal may be configured to reproducecontent, to obtain metadata corresponding to a predetermined portion ofthe content, to select, based on the metadata, at least one of the atleast one external device to perform a function associated with thepredetermined portion of the content, and to transmit a control commandto the at least one external device. The at least one external device isconfigured to receive the control command, and to perform the functionassociated with the predetermined portion of the content.

In accordance with another aspect of the present disclosure, the systemmay include a gateway configured to connect the terminal and the atleast one external device.

In accordance with another aspect of the present disclosure, the gatewaymay be configured to mediate communication between the terminal and theat least one external device so as to convert the control commandtransmitted by the terminal to a protocol of the at least one externaldevice.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a diagram of a content reproduction system according to anembodiment of the present disclosure;

FIG. 2 is a flowchart of a method of connecting an external device, themethod being performed by a terminal, according to an embodiment of thepresent disclosure;

FIG. 3 is a flowchart of a method of reproducing content, according toan embodiment of the present disclosure;

FIGS. 4A and 4B illustrate a configuration of metadata, according to anembodiment of the present disclosure;

FIG. 5 is a flowchart of a method of obtaining metadata, according to anembodiment of the present disclosure;

FIG. 6 illustrates a Graphical User Interface (GUI) for insertingmetadata, according to an embodiment of the present disclosure;

FIG. 7 is a flowchart of a method of generating metadata, according toan embodiment of the present disclosure;

FIG. 8 illustrates an example of an image processing algorithm accordingto an embodiment of the present disclosure;

FIG. 9 is a flowchart of a method of generating metadata correspondingto a predetermined portion of content, according to an embodiment of thepresent disclosure;

FIG. 10 is a flowchart of a method of executing an application,according to an embodiment of the present disclosure;

FIGS. 11A and 11B illustrate diagrams related to a method of reproducinganimation content by interoperation of a terminal with an externaldevice, according to an embodiment of the present disclosure;

FIG. 12 is a diagram related to a method of reproducing movie content byinteroperation of a terminal with an external device, according to anembodiment of the present disclosure;

FIG. 13 illustrates an example of a GUI that displays a list of externaldevices connected to a terminal according to an embodiment of thepresent disclosure;

FIG. 14 is a flowchart of a method of reproducing content byinteroperation of a terminal with at least one external device,according to an embodiment of the present disclosure;

FIGS. 15A and 15B illustrate screens that reproduce education content,according to an embodiment of the present disclosure;

FIGS. 16A and 16B illustrate screens that reproduce education content,according to an embodiment of the present disclosure;

FIGS. 17A and 17B illustrate screens that reproduce cooking classcontent while a terminal interoperates with an external device,according to an embodiment of the present disclosure;

FIG. 18 is a diagram of a content reproduction system, according to anembodiment of the present disclosure;

FIG. 19 is a flowchart of a method of obtaining metadata via a gateway,according to an embodiment of the present disclosure;

FIG. 20 is a flowchart of a method of converting a control command via agateway, according to an embodiment of the present disclosure; and

FIG. 21 is a block diagram of a terminal, according to an embodiment ofthe present disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the present disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thepresent disclosure. In addition, descriptions of well-known functionsand constructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of the presentdisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of the presentdisclosure is provided for illustration purpose only and not for thepurpose of limiting the present disclosure as defined by the appendedclaims and their equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

In addition, when a part “includes” or “comprises” an element, unlessthere is a particular description contrary thereto, the part may furtherinclude other elements, not excluding the other elements. In thefollowing description, terms such as “unit” and “module” indicate a unitfor processing at least one function or operation, wherein the unit andthe block may be embodied as hardware or software or may be embodied bycombining hardware and software.

Throughout the specification, the term “application” means a group ofcomputer programs designed to perform particular tasks. In thespecification, the application may be of different types. For example,the application may be, but is not limited to, a game application, amusical instrument play application, a moving picture reproductionapplication, a map application, a broadcasting application, an exercisesupport application, a payment application, and/or the like.

Throughout the specification, the term “topology” means the arrangementof devices (or nodes). The topology may include physical topology andlogical topology. For example, the topology may be defined according tophysical or logical connection between the devices that belong to atopology structure. For example, different topologies may be defined byat least one of a cooperative relationship between the devices, a methodof connecting the devices, a data transmission speed between thedevices, a flow of data exchanged between the devices, a type of asignal exchanged between the devices, and a type of an applicationinstalled in each of the devices.

In addition, throughout the specification, a host terminal, a mainexternal device, and a sub external device may be determined accordingto a position of devices, and an absolute or relative role of thedevices in the topology structure. Thus, a predetermined device mayoperate as at least one of the host terminal, the main external device,and the sub external device in the topology structure.

Various embodiments of the present disclosure will now be described morefully with reference to the accompanying drawings. However, variousembodiments of the present disclosure may be embodied in many differentforms, and should not be construed as being limited to the variousembodiments set forth herein, Rather, the various embodiments describedherein are provided so that the present disclosure is thorough andcomplete, and will fully convey the concept of the various embodimentsof the present disclosure to those of ordinary skill in the art. In thefollowing description, well-known functions or constructions are notdescribed in detail because descriptions thereof may obscure variousembodiments of the present disclosure with unnecessary detail. In thefollowing description, reference numerals in the drawings denote like orsimilar elements throughout the specification.

As used herein, the term “and/or” includes any and all combinations ofone or more of the associated listed items. Expressions such as “atleast one of,” when preceding a list of elements, modify the entire listof elements and do not modify the individual elements of the list.

FIG. 1 is a diagram of a content reproduction system according to anembodiment of the present disclosure.

Referring to FIG. 1, the content reproduction system may include aterminal 100 and an external device 200. However, not all shown elementsare necessary elements. For example, the content reproduction system maybe embodied with more or less elements than the shown elements. Forexample, as illustrated in FIG. 18 and as described in relation thereto,the content reproduction system may further include a gateway to connectthe terminal 100 and the external device 200.

According to various embodiments of the present disclosure, the terminalmay connect to a combination of external devices 200-1 to 200-N.

The terminal 100 may be connected to the external device 200 in a wiredor wireless manner. In particular, according to various embodiments ofthe present embodiment, the terminal 100 may communicate with theexternal device 200 via short-distance communication. Examples of theshort-distance communication may include, but are not limited to, awireless Local Area Network (LAN) (e.g., Wi-Fi), Near FieldCommunication (NFC), Bluetooth, Bluetooth Low Energy (BLE), ZigBee,Wi-Fi Direct (WFD), Ultra WideBand (UWB), and/or the like.

The terminal 100 may control the external device 200 by executing anapplication related to the external device 200. In addition, when theapplication is executed, the terminal 100 may automatically scan theexternal device 200 and may connect the terminal 100 to the externaldevice 200. For example, according to various embodiments of the presentembodiment, the terminal 100 may be connected to the external device 200by executing a content reproduction application or a control applicationfor controlling the external device 200.

In addition, the terminal 100 may reproduce predetermined content alongwith the external device 200 by using the content reproductionapplication or the control application for controlling the externaldevice 200.

The terminal 100 may be variously embodied. For example, according tovarious embodiments of the present embodiment, the terminal 100 may be,but is not limited to, a mobile phone, a smartphone, a laptop computer,a tablet Personal Computer (PC), an MP3 player, a digital camera, aportable game console, a Global Positioning System (GPS) navigation, anelectronic book terminal, a terminal for digital broadcasting, aPersonal Digital Assistant (PDA), a Portable Multimedia Player (PMP), anavigation device, a smart TV, Consumer Electronic (CE) devices (e.g., arefrigerator, an air conditioner, or the like that have a displaypanel), and the like.

As disclosed above, the application that is executed in the terminal 100may be related to the external device 200. For example, the contentreproduction application or the control application may be related tothe external device 200. According to various embodiments of the presentdisclosure, the external device 200 may be a single device or may bemultiple devices.

The external device 200 may communicate with the terminal 100 viashort-distance communication. In this case, the external device 200 mayreceive a control command from the terminal 100 via the short-distancecommunication. In addition, the external device 200 may transmit Inputand Output (I/O) data (e.g., a message) to the terminal 100 via theshort-distance communication.

The external device 200 may include a short-distance communicationmodule. For example, the external device 200 may include a BLE module,an NFC module, and/or the like. The external device 200 may receive aconnection request from the terminal 100 via BLE and/or NFC and then mayactivate a Wi-Fi module or a Bluetooth module. In addition, the externaldevice 200 may transmit authentication information, connectioninformation, or the like to the terminal 100 via BLE and/or NFC. Theexternal device 200 may receive a control command from the terminal 100and may perform the control command.

The external device 200 may be variously embodied. The external device200 may include an input device, an output device, a control device, orthe like. For example, the external device 200 may include, but is notlimited to, a microphone, a speaker, a pedal, a joy stick, a musicalinstrument (e.g., a piano, an organ, an electric keyboard, a guitar, aviolin, a cello, or the like), a game control device, a doll, a medicalequipment, a sporting equipment, CE devices (e.g., an air-conditioner,an oven, a refrigerator, an electric fan, or the like), and the like.

Hereinafter, a method of registering and connecting the external device200, the method performed by the terminal 100, will now be describedwith reference to FIG. 2.

FIG. 2 is a flowchart of a method of connecting the external device 200,the method performed by the terminal 100 according to an embodiment ofthe present disclosure.

Referring to FIG. 2, at operation S210, the terminal 100 may detectoccurrence of first short-distance communication. According to variousembodiments of the present disclosure, the first short-distancecommunication may be, but is not limited to, NFC and BLE communication.

According to various embodiments of the present disclosure, if the firstshort-distance communication is NFC-based communication, when theexternal device 200 is positioned within a range of the NFC-basedcommunication, the terminal 100 may detect the occurrence of the firstshort-distance communication. According to various embodiments of thepresent disclosure, if the first short-distance communication isBLE-based communication, when the terminal 100 receives a signalbroadcasted from the external device 200 via Bluetooth communication,the terminal 100 may detect the occurrence of the first short-distancecommunication.

At operation S220, the terminal 100 may receive device information aboutthe external device 200 via the first short-distance communication. Thedevice information about the external device 200 may include at leastone of IDentification (ID) information of the external device 200,connection information for second short-distance communication, a typeof the external device 200, information about a capability of theexternal device 200, information about a category and a protocol used tocontrol the external device 200, and/or the like. According to variousembodiments of the present disclosure, the capability of the externaldevice 200 may be, but is not limited to, a supportable communicationcapability, an audio output capability, a video output capability, avoice recording capability, an image-capturing capability, a winddirection adjusting capability, a humidity detecting capability, and/orthe like.

According to various embodiments of the present disclosure, the secondshort-distance communication may be, but is not limited to, wireless LANcommunication and Bluetooth communication. The Wi-Fi may include aninfrastructure mode using an Access Point (AP), a wireless router, orthe like, and an ad hoc mode in which terminals exchange data by using aPeer-To-Peer (P2P) method without using an AP.

Thus, the connection information for the second short-distancecommunication may include a communication method performed by theexternal device 200, connection information (a Service Set IDentifier(SSID), an Internet Protocol (IP) address, a Media Access Control (MAC)address, a channel number, a security key, or the like) of the wirelessLAN used by the external device 200, a MAC address of the externaldevice 200, a Bluetooth address of the external device 200, a productname of the external device 200, profile information of the externaldevice 200, or the like. The communication method may be the wirelessLAN (the ad hoc mode or the infrastructure mode), Bluetooth, ZigBee,WFD, UWB, or the like.

According to various embodiments of the present disclosure, the deviceinformation about the external device 200 may further includeinformation about an application related to the external device 200. Theinformation about the application may include, but is not limited to, IDinformation of the application (e.g., an application ID, an applicationtitle, and an application classification code), version information ofthe application, and link information of the application (e.g.,connection information of a web page for downloading the application).

At operation S230, the terminal 100 may search for an application. Forexample, the terminal 100 may search for the application related to theexternal device 200.

At operation S240, the terminal 100 may determine, based on theinformation about the application received from the external device 200,whether the application related to the external device 200 waspreviously installed in the terminal 100.

If the terminal 100 determines that the application related to theexternal device 200 was installed in terminal 100 at operation S240,then the terminal 100 may proceed to operation S260.

In contrast, when the terminal 100 determines that the applicationrelated to the external device 200 was not installed in the terminal 100at operation S240, the terminal 100 may proceed to operation S250 atwhich the terminal 100 may download and install the application relatedto the external device 200.

At operation S260, the terminal 100 may execute the application.

At operation S270, the terminal 100 may perform the secondshort-distance communication. For example, when the secondshort-distance communication is Bluetooth communication, the terminal100 may activate the Bluetooth communication module, and when the secondshort-distance communication is wireless LAN communication, the terminal100 may activate the Wi-Fi module. Afterward, the terminal 100 mayrequest the external device 200 for connection. For example, theterminal 100 may request that the external device 200 connect to theterminal 100 via the second short-distance communication.

At operation S280, when the external device 200 accepts the connection,the terminal 100 and the external device 200 may communicate with eachother via the second short-distance communication.

At operation S290, the terminal 100 may store and manage the deviceinformation about the external device 200 in a memory. For example, whenthe device information about the external device 200 is changed, theterminal 100 may update the device information stored in the memory.

According to various embodiments of the present disclosure, the terminal100 may scan the external device 200 in a Universal Plug and Play (UPnP)manner. For example, when a new external device is connected to anetwork, the terminal 100 may perform communication with the newexternal device and then may receive device information about the newexternal device (e.g., ID information, connection information forconnecting second short-distance communication, information about anapplication related to the new external device, or the like) from thenew external device. The terminal 100 may execute the applicationrelated to the new external device and may communicate with the newexternal device via the second short-distance communication.

According to various embodiments of the present disclosure, operationsS210 through S290 may be performed before the terminal 100 reproducescontent or may be performed in an early period during reproduction ofthe content. According to various embodiments of the present disclosure,operations S210 through S290 may be performed while the terminal 100reproduces the content. For example, when the external device 200 joinsa network while the terminal 100 reproduces the content, or when theterminal 100 detects occurrence of first short-distance communicationperformed by the external device 200, the terminal 100 may performoperations S220 through S290 during reproduction of the content.

According to various embodiments of the present disclosure, an order ofoperations S210 through S290 may be changed or some operations may beskipped.

In the embodiment illustrated in FIG. 2, the terminal 100 and theexternal device 200 communicate via the short-distance communication(e.g., Wi-Fi or Bluetooth). However, various embodiments of the presentdisclosure are not limited thereto. For example, according to variousembodiments of the present disclosure, the terminal 100 and the externaldevice 200 may communicate to each other in a wired manner via a mobilecommunication network or another short-distance communication (e.g.,WFD, UWB, ZigBee, or the like).

Hereinafter, a method of reproducing a predetermined portion of contentwhile the terminal 100 and the external device 200 are wiredly orwirelessly connected will now be described in detail with reference toFIG. 3.

FIG. 3 is a flowchart of a method of reproducing content according to anembodiment of the present disclosure.

At operation S310, the terminal 100 may obtain metadata corresponding toa predetermined portion of the content.

According to various embodiments of the present disclosure, the terminal100 may extract the metadata corresponding to the predetermined portionwhich was previously inserted into the content. According to variousembodiments of the present disclosure, the terminal 100 may generate themetadata corresponding to the predetermined period by performing imageprocessing on the content. According to various embodiments of thepresent disclosure, the terminal 100 may receive the metadatacorresponding to the predetermined portion of the content from anexternal source, separately from the content.

According to various embodiments of the present disclosure, the contentmay include multimedia content. For example, the content may include,but is not limited to, education content, movie content, broadcastingcontent, and game content, and/or the like.

According to various embodiments of the present disclosure, thepredetermined portion of the content indicates a reproduction portion ofthe content defined by a time period. The predetermined portion may beformed of at least one frame, at least one sequence, or at least onescene. However, various embodiments of the present disclosure are notlimited thereto.

According to various embodiments of the present disclosure, the metadatacorresponding to the predetermined portion may mean information used toreproduce the predetermined portion of the content. According to variousembodiments of the present disclosure, the metadata may include at leastone of information about the predetermined period (e.g., a time stamp,an index, or the like), ID information of at least one external deviceto reproduce the predetermined period, and capability information abouta capability to be performed by the at least one external device. Themetadata is described below with reference to FIGS. 4A and 4B.

FIGS. 4A and 4B illustrate configuration of metadata according to anembodiment of the present disclosure.

Referring to FIG. 4A, metadata 400 may include information 410indicating a predetermined portion, and control information 420 forcontrolling the external device 200. The control information 420 mayinclude ID information 421 of the external device 200 to reproduce thepredetermined portion, capability information 422 about a capability tobe performed by the external device 200, and additional information 423about the capability.

Referring to FIG. 4B, in a case in which a speaker and a robot cleanerare used to reproduce a predetermined portion of content (e.g., TimeStamp: 930000), the metadata 400 that corresponds to the predeterminedportion (e.g., Time Stamp: 930000) may include control information 420-1about the speaker and control information 420-2 about the robot cleaner.

The control information 420-1 about the speaker may include IDinformation used to identify the speaker (e.g., device category:speaker), capability information about a capability to be performed bythe speaker (e.g., a sound output), and additional information about thecapability (e.g., data of sound to be output from the speaker).

In addition, the control information 420-2 about the robot cleaner mayinclude ID information used to identify the robot cleaner (e.g., devicecategory: robot cleaner), capability information about a capability tobe performed by the robot cleaner (e.g., a start of cleaning), andadditional information about the capability (e.g., auto-mode setting).

Referring back to FIG. 3, at operation S320, the terminal 100 may selectat least one external device 200 to reproduce the predetermined portionof the content, based on the obtained metadata corresponding to thepredetermined portion.

For example, as illustrated in FIG. 4B, when the control information420-1 about the speaker and the control information 420-2 about therobot cleaner are included in the metadata 400 corresponding to thepredetermined portion (e.g., Time Stamp: 930000), the terminal 100 mayselect the speaker and the robot cleaner as the external devices 200 toreproduce the predetermined portion.

The speaker and the robot cleaner may be previously connected to theterminal 100 before reproduction of the predetermined portion of thecontent. In addition, the speaker and the robot cleaner may be in astandby state in which the speaker and the robot cleaner may immediatelyperform a control command of the terminal 100.

For example, the terminal 100 may select at least one external device200 that is to reproduce the predetermined portion and that is fromamong a plurality of external devices physically connected to theterminal 100. The physical connection may include active connection,passive connection, remote connection, wired connection, wirelessconnection, short-distance communication connection, or the like.

According to various embodiments of the present disclosure, the terminal100 may select the external device 200 to reproduce the predeterminedportion, and may scan the selected external device 200 in an UPnP way.

In a case in which the metadata corresponding to the predeterminedportion does not include the ID information of the external device 200but only includes capability information about a capability to beperformed by the external device 200, the terminal 100 may select the atleast one external device 200 by using a capability mapping table. Thecapability mapping table may be a table in which a particular capabilityis mapped with the ID information of the external device 200 thatperforms the particular capability.

For example, when the metadata corresponding to the predeterminedportion includes only capability information indicating “sound output”,the terminal 100 may select by using the capability mapping table a‘speaker’ that performs “sound output” as the external device 200 toreproduce the predetermined portion of the content.

At operation S330, the terminal 100 may transmit a control command tothe selected external device 200. The terminal 100 may transmit thecontrol command to the selected external device 200 via wired orwireless communication.

The terminal 100 may convert control information included in themetadata into the control command that matches a control protocol of theselected external device 200.

At operation S340, the terminal 100 may reproduce the predeterminedportion of the content by interoperating with the selected externaldevice 200. For example, the terminal 100 and the selected externaldevice 200 may reproduce together the predetermined portion of thecontent. When the control command is transmitted to the selectedexternal device 200, the terminal 100 may discontinue reproduction ofthe predetermined portion of the content, and when the terminal 100receives a message corresponding to the control command from theselected external device 200, the terminal 100 may resume reproductionof the discontinued content.

Thus, the terminal 100 may provide a new experience of contentreproduction to a user by interoperating with the selected externaldevice 200.

FIG. 5 is a flowchart of a method of obtaining metadata, according to anembodiment of the present disclosure.

Referring to FIG. 5, at operation S510, the terminal 100 may obtaincontent to which metadata is inserted.

For example, the terminal 100 may receive from an external source thecontent to which the metadata is inserted by a content producer or acontent provider. In addition, the terminal 100 may obtain from a memorycontent to which metadata is inserted directly by a user who uses thecontent. A method of inserting metadata by the content producer, thecontent provider, or the user will be described in detail with referenceto FIG. 6.

At operation S520, the terminal 100 may extract the metadatacorresponding to a predetermined portion which was previously insertedinto the content.

The terminal 100 may extract the metadata corresponding to thepredetermined portion before the content is reproduced. According tovarious embodiments of the present disclosure, when the terminal 100receives a content reproduction start command, the terminal 100 mayrecognize a plurality of pieces of metadata corresponding to allportions of the content and may perform pre-processing in an earlyportion during reproduction of the content. According to variousembodiments of the present disclosure, the terminal 100 may extract themetadata corresponding to the predetermined portion while the content isreproduced.

According to various embodiments of the present disclosure, the terminal100 may receive a metadata file with respect to all portions of thecontent, separately from the content.

At operation S530, the terminal 100 may select at least one externaldevice 200 to reproduce the predetermined portion of the content, basedon the extracted metadata.

At operation S540, the terminal 100 may transmit a control command tothe at least one external device 200. For example, at operation S540,the terminal may transmit the control command to the selected at leastone external device 200.

At operation S550, the terminal 100 may reproduce the predeterminedportion of the content by interoperating with the at least one externaldevice 200.

Operations S530 through S550 correspond to operations S320 through S340of the flowchart in FIG. 3, and thus, detailed descriptions thereof areomitted here.

FIG. 6 illustrates a Graphical User Interface (GUI) for insertingmetadata 600, according to an embodiment of the present disclosure.

Referring to FIG. 6, the GUI for inserting metadata 600 may include apreview field 610 for displaying a preview image corresponding to apredetermined portion of content, a time line field 620 for selectingthe predetermined portion, a metadata list field 630 for displaying alist of metadata corresponding to the predetermined portion, and acontrol field 640 for adding control information.

A content producer, a content provider, or a content user may generatethe metadata corresponding to the predetermined portion of the contentby using the GUI for inserting metadata 600. For example, the contentuser may select the predetermined portion of the content in the timeline field 620. Thereafter, the preview field 610 may display thepreview image corresponding to the predetermined portion selected by thecontent user.

The content user may input control information corresponding to thepredetermined portion via the control field 640. For example, thecontent user may select a type of at least one external device toreproduce the predetermined portion via a device category 641, may inputa setting for a capability to be performed by the selected at least oneexternal device via a control command 642 and may input optional data643, and then may select an ‘add’ button 644. Thereafter, the metadatacorresponding to the predetermined portion may be generated and added tothe metadata list field 630.

FIG. 7 is a flowchart of a method of generating metadata, according toan embodiment of the present disclosure.

At operation S710, the terminal 100 may perform image processing oncontent. The terminal 100 may perform image processing on the content byusing various image processing algorithms (e.g., as illustrated in FIG.8).

The terminal 100 may recognize an atmosphere, a feeling, a device, orthe like that appears in a predetermined portion of the content byperforming image processing on the content.

According to various embodiments of the present disclosure, the terminal100 may analyze audio data included in the content.

At operation S720, the terminal 100 may generate a plurality of piecesof metadata corresponding to portions of the content, respectively,based on a result of the image processing. For example, the terminal 100may previously generate the plurality of pieces of metadatacorresponding to all of the portions of the content before the contentis reproduced. The terminal 100 may previously turn on a power of atleast one external device 200 to reproduce the content or may previouslyestablish communication with the at least one external device 200, basedon the plurality of pieces of metadata corresponding to the all portionsof the content.

The terminal 100 may generate metadata based on a type of the at leastone external device 200 connected to the terminal 100. For example, in acase in which a device A, a device B, and a device C are connected tothe terminal 100 via wired or wireless communication, the terminal 100may generate metadata so as to use the device A, the device B, and thedevice C when a predetermined portion of content is reproduced. Forexample, the terminal 100 may not include control information about adevice D, which is not connected to the terminal 100, in the metadata.In addition, when the terminal 100 detects connection with anotherexternal device during reproduction of the content, the terminal 100 maygenerate metadata corresponding to the predetermined portion, inconsideration of the connected other external device.

The terminal 100 may insert the metadata generated by performing imageprocessing into the content, or may manage the metadata as a separatemetadata file, separately from the content.

At operation S730, the terminal 100 may start reproduction of thecontent.

At operation S740, the terminal 100 may identify control information byusing the metadata corresponding to the predetermined portion of thecontent. For example, the terminal 100 may identify thepreviously-generated metadata corresponding to the predetermined portionwhile the terminal 100 reproduces the content.

The terminal 100 may select the at least one external device 200 toreproduce the predetermined portion of the content, and may identify,based on the control information included in the metadata, capabilityinformation about a capability operation to be performed by the at leastone external device 200.

At operation S750, the terminal 100 may generate a control command byusing the control information included in the metadata corresponding tothe predetermined portion of the content. For example, the terminal 100may convert the capability information about the capability to beperformed by the at least one external device 200, which is included inthe control information, into a control command matching a controlprotocol of the at least one external device 200.

At operation S760, the terminal 100 may transmit the control command tothe at least one external device 200. The control protocol may includethe capability information about the capability (e.g., a sound output,and/or the like) to be performed by the at least one external device200, temporal information about a time (e.g., after 1 minute, and/or thelike) when the capability is to be performed by the at least oneexternal device 200, or the like.

At operation S770, the terminal 100 may reproduce the predeterminedportion of the content by interoperating with the at least one externaldevice 200. For example, at operation S780, when the terminal reproducesthe predetermined portion of the content at S770, the at least oneexternal device 200 may perform the control command.

FIG. 8 illustrates an example of an image processing algorithm accordingto an embodiment of the present disclosure.

Referring to FIG. 8, the terminal 100 may divide a video clip, which iscommonly expressed as a scene, in units of shots, may obtain a key framefrom each of the shots, and then may extract 5 characteristics of anaverage color histogram, an average brightness, an average edgehistogram, an average shot duration, and a gradual shot change rate fromthe shots in the scene.

A color is the most basic characteristic that expresses contents of avideo, and is effective in expressing a particular feeling. For example,a warm mood image mainly includes red-component elements, and a coldmood image mainly includes blue-component elements. In a case of avideo, an “action” scene generally includes red and purple colors, and a“quiet” scene generally includes blue, green, and white colors.

A bright mood image expresses light and happy feelings, whereas a darkimage expresses rigid, heavy, and gloomy feelings. In the video, the“quiet” scene mainly includes information about bright-tone colors.

A gloomy mood image includes a small number of main edges as the imageis entirely blurred, and a happy mood image includes a relatively greatnumber of main edges.

In “action”, “exciting”, scenes or the like, shots are rapidly changed,and in “quiet”, “relaxed”, and “happy” scenes or the like a length ofeach shot is long and variation in each shot is not great.

In the video, a boundary between shots due to a gradual change mayarouse a particular feeling. The “quiet” scene may generally includevariation of gradual shots such as dissolve.

According to various embodiments of the present disclosure, the terminal100 may analyze a feeling of each shot by using the extracted 5characteristics (e.g., the average color histogram, the averagebrightness, the average edge histogram, the average shot duration, andthe gradual shot change rate).

According to various embodiments of the present disclosure, the terminal100 may extract a characteristic element from each shot by using an edgedetection method, a frequency-based method, a template-based method, orthe like. For example, the terminal 100 may recognize that a sceneincludes a vehicle by using details such that the vehicle is verticallyand horizontally symmetrical, a zone below the vehicle is darkly shaded,a rear portion of the vehicle generally has a quadrangular shape withmany edges, or the like.

According to various embodiments of the present disclosure, the terminal100 may generate the metadata by using various image processingalgorithms, other than the aforementioned image processing algorithm.Because the image processing algorithms are well known to ordinaryskilled in the art, detailed descriptions thereof are omitted here.

FIG. 9 is a flowchart of a method of generating metadata correspondingto a predetermined portion of content according to an embodiment of thepresent disclosure.

Referring to FIG. 9, at operation S905, the terminal 100 may startreproduction of the content. For example, when the terminal 100 receivesa content reproduction start command from a user, the terminal may startreproduction of the content by using a content reproduction application.

At operation S910, the terminal 100 may perform image processing on ann^(th) portion of the content. For example, the terminal 100 may performimage processing on a predetermined portion of the content in real-time,separately from reproduction of the content.

By performing image processing on the n^(th) portion, the terminal 100may extract device information about at least one external device 200related to the n^(th) portion, capability information about a capabilityto be performed by the at least one external device 200 duringreproduction of the n^(th) portion, or the like.

At operation S915, the terminal 100 may generate metadata correspondingto the n^(th) portion of the content. For example, the terminal 100 maygenerate the metadata corresponding to the n^(th) portion by using thedevice information about the at least one external device 200 related tothe n^(th) portion and the capability information about the capabilityto be performed by the at least one external device 200 duringreproduction of the n^(th) portion which are extracted by performingimage processing on the n^(th) portion.

At operation S920, the terminal 100 may identify control informationincluded in the metadata corresponding to the n^(th) portion.

At operation S925, the terminal 100 may convert the control informationinto a control command that matches a control protocol of the at leastone external device 200.

At operation S930, the terminal 100 may transmit the control command tothe at least one external device 200. According to various embodimentsof the present disclosure, the terminal 100 may transmit the controlcommand to the at least one external device 200 via short-distancecommunication.

At operation S935, the terminal 100 may reproduce a predeterminedportion of the content by interoperating with the at least one externaldevice 200.

At operation S940, the at least one external device 200 may perform thecontrol command. For example, the terminal 100 and the at least oneexternal device 200 may reproduce together the predetermined portion ofthe content.

Operations S920 through S940 correspond to operations S740 through S780of the flowchart in FIG. 7, and thus, detailed descriptions thereof areomitted here.

At operation S945, the terminal 100 may check (e.g., determine) whetherreproduction of the content is complete.

If the terminal 100 determines that reproduction of the content is notcomplete at operation S945, then the terminal 100 may proceed tooperation S950 at which the terminal 100 may perform image processing ona next portion (e.g., an n+1^(th) portion) and then may generatemetadata corresponding to the next portion (the n+1^(th) portion). Forexample, operations S910 through S945 may be repeatedly performed.

In contrast, if the terminal 100 determines that reproduction of thecontent is complete at operation S945, then the terminal 100 may proceedto operation S955 at which the terminal 100 may check (e.g., determine)whether the content is deleted according to a deletion request by theuser.

If the terminal 100 determines that the content is not deleted atoperation S955, then the terminal 100 may proceed to operation S960 atwhich the terminal 100 may store the metadata generated at operationsS910 through S915. In this case, the terminal 100 may store the metadataby inserting the metadata into the content or may store the metadata asa separate metadata file.

In contrast, if the terminal 100 determines that reproduction of thecontent is complete at operation S945, and that the content is deletedat operation S955, then the terminal 100 may proceed to operation S965at which the terminal 100 may delete the metadata generated atoperations S910 through S915 and may therefore efficiently manage amemory.

According to various embodiments of the present disclosure, regardlessof deletion of the content, when reproduction of the content iscomplete, the terminal 100 may delete the metadata generated byperforming image processing.

FIG. 10 is a flowchart of a method of executing an application accordingto an embodiment of the present disclosure.

At operation S1010, the terminal 100 may execute a first application soas to reproduce content.

At operation S1020, the terminal 100 may obtain metadata correspondingto a predetermined portion of the content via the first application. Thefirst application may extract the metadata corresponding to thepredetermined portion, which is inserted into the content, or maygenerate the metadata corresponding to the predetermined portion byperforming image processing on the content. In addition, the firstapplication may request and receive the metadata corresponding to thepredetermined portion from an external server.

At operation S1030, the terminal 100 may identify control informationincluded in the metadata corresponding to the predetermined portion viathe first application. The first application may transmit to a secondapplication the control information included in the metadatacorresponding to the predetermined portion.

At operation S1040, the terminal 100 may execute the second applicationso as to control at least one external device 200.

At operation S1050, the terminal 100 may generate a control command byusing the second application. The second application may haveinformation about a control protocol of the at least one external device200. Thus, the second application may convert the control informationreceived from the first application into the control command thatmatches the control protocol of the at least one external device 200.

At operation S1060, the terminal 100 may transmit the control command tothe at least one external device 200.

At operation S1070, the terminal 100 may reproduce the predeterminedportion of the content by interoperating with the at least one externaldevice 200. For example, at operation S1080, the at least one externaldevice 200 may perform the control command when the terminal 100reproduces the predetermined portion of the content.

According to various embodiments of the present disclosure, the firstapplication and the second application may be sub-applications of a mainapplication. In addition, the first application and the secondapplication may be embodied as separate function blocks in oneapplication.

FIGS. 11A and 11B illustrate diagrams related to a method of reproducinganimation content by interoperation of a terminal with an externaldevice according to an embodiment of the present disclosure.

Referring to FIG. 11A, the terminal 100 may reproduce the animationcontent. The terminal 100 may obtain metadata corresponding to an n^(th)portion of the animation content and may identify control informationincluded in the metadata.

For example, when an X character and a Y character appear in the n^(th)portion of the animation content, the metadata corresponding to then^(th) portion may include the metadata for controlling an X characterdoll 1110 and a Y character doll 1120.

The terminal 100 may transmit control commands to the X character doll1110 and the Y character doll 1120, respectively, based on the controlinformation. Thereafter, the terminal 100, the X character doll 1110,and the Y character doll 1120 may reproduce together the n^(th) portionof the animation content.

Referring to FIG. 11B, the terminal 100 may reproduce an n+1^(th)portion of the animation content. In this case, the terminal 100 mayobtain metadata corresponding to the n+1^(th) portion and may identifycontrol information included in the obtained metadata. In a case inwhich a Z character as well as the X character and the Y characterappear in the n+1^(th) portion of the animation content, the metadatacorresponding to the n+1^(th) portion may include the controlinformation for controlling the X character doll 1110, the Y characterdoll 1120, and a Z character doll 1130.

The terminal 100 may transmit control commands to the X character doll1110, the Y character doll 1120, and the Z character doll 1130,respectively, based on the control command. Thereafter, the terminal100, the X character doll 1110, the Y character doll 1120, and the Zcharacter doll 1130 may reproduce together the n+1^(th) portion of theanimation content.

Therefore, according to various embodiments of the present disclosure,the terminal 100 and character dolls appearing in scenes of theanimation content may reproduce together the animation content.

FIG. 12 is a diagram related to a method of reproducing movie content byinteroperation of a terminal with an external device according to anembodiment of the present disclosure.

Referring to FIG. 12, the terminal 100 may reproduce the movie content.The terminal 100 may generate metadata corresponding to a predeterminedportion of the movie content by performing image processing on the moviecontent. For example, if the predetermined portion of the movie contentincludes a whirlwind scene, the terminal 100 may generate the metadataincluding a plurality of pieces of control information related to ‘airconditioner 1210: start cooling’, ‘audio system 1220: output sound’,‘tablet PC 1230: generate strong vibration and output sound’, ‘robotcleaner 1240: start moving and strongly suck up’, or the like. In thiscase, the air conditioner 1210, the audio system 1220, the tablet PC1230, and the robot cleaner 1240 may be previously connected to theterminal 100 in a wired or wireless manner.

Before the terminal 100 reproduces the predetermined portion of themovie content, the terminal 100 may transmit control commands to the airconditioner 1210, the audio system 1220, the tablet PC 1230, and therobot cleaner 1240, respectively. The control commands may include aplurality of pieces of capability information about capabilities to beperformed by the air conditioner 1210, the audio system 1220, the tabletPC 1230, and the robot cleaner 1240, respectively.

Thereafter, when the terminal 100 reproduces the whirlwind scene, theair conditioner 1210, the audio system 1220, the tablet PC 1230, and therobot cleaner 1240 may perform the control commands, respectively, whichare received from the terminal 100. Therefore, according to variousembodiments of the present disclosure, the terminal 100 may providelifelike reproduction of the movie content by interoperating with one ormore external devices 200.

FIG. 13 illustrates an example of a GUI that displays a list of externaldevices connected to a terminal according to an embodiment of thepresent disclosure.

Referring to FIG. 13, according to a user request, the terminal 100 maydisplay information (e.g., device ID values, capabilities, and/or thelike) about the external devices connected to the terminal 100. Forexample, the terminal 100 may display via a GUI the list of the externaldevices connected to the terminal 100.

As illustrated in FIG. 13, when a speaker 1310, a robot cleaner 1320,and an air conditioner 1330 are wiredly or wirelessly connected to theterminal 100, the terminal 100 may provide an external-device listincluding ID information of each of the speaker 1310, the robot cleaner1320, and the air conditioner 1330.

In this case, a user may select from the external-device list one ormore external devices to be used for reproducing content and otherexternal devices not to be used for reproducing content. For example,the user may set the speaker 1310 and the robot cleaner 1320 to be usedfor reproducing content, and set the air conditioner 1330 not to be usedfor reproducing content.

In this case, the terminal 100 may transmit control commands only to thespeaker 1310 and the robot cleaner 1320 that are selected by the user tobe used for reproducing content, and may not transmit a control commandto the air conditioner 1330.

In addition, when the terminal 100 generates metadata by performingimage processing, the terminal 100 may generate the metadata includingcontrol information only about the speaker 1310 and the robot cleaner1320.

FIG. 14 is a flowchart of a method of reproducing content byinteroperation of a terminal with at least one external device accordingto an embodiment of the present disclosure.

At operation S1400, the terminal 100 and the at least one externaldevice 200 may communicate with each other via short-distancecommunication. For example, the terminal 100 and the at least oneexternal device 200 may communicate with each other via Wi-Fi orBluetooth. For example, at operation S1400, the terminal 100 may connectwith at least one external device 200.

At operation S1410, the terminal 100 may start reproduction of thecontent in response to a user input.

At operation S1420, the terminal 100 may obtain metadata correspondingto a predetermined portion of the content.

At operation S1430, the terminal 100 may identify control information byusing the metadata. For example, the terminal 100 may identify thecontrol information included in the metadata corresponding to thepredetermined portion of the content during reproduction of the content.By referring to the control information included in the metadata, theterminal 100 may select the at least one external device 200 toreproduce the predetermined portion of the content, and may generate acontrol command by using capability information about a capability to beperformed by the at least one external device 200.

At operation S1440, the terminal 100 may transmit the control command tothe at least one external device 200.

At operation S1450, the terminal 100 may discontinue reproduction of thecontent. For example, when the control command is transmitted to the atleast one external device 200, the terminal 100 may discontinuereproduction of the content.

According to various embodiments of the present disclosure, the controlcommand may be, but is not limited to, a command for controlling the atleast one external device 200 to display a question included in thecontent or a command for controlling the at least one external device200 to set itself for reproducing content by using a setting valueincluded in the content.

At operation S1460, the at least one external device 200 may execute thecontrol command received from the terminal 100.

According to various embodiments of the present disclosure, operationsS1450 and S1460 may be simultaneously performed or operation S1460 maybe performed before operation S1450.

At operation S1470, the terminal 100 may receive a message correspondingto the control command from the at least one external device 200. Themessage corresponding to the control command may include a responsemessage with respect to the question included in the content, a settingcompletion message, or the like.

At operation S1480, the terminal 100 may check the message received fromthe at least one external device 200.

At operation S1490, the terminal 100 may resume reproduction of thediscontinued content. For example, when the terminal 100 receives themessage corresponding to the control command from the at least oneexternal device 200, the terminal 100 may resume reproduction of thediscontinued content.

According to various embodiments of the present disclosure, at operationS1480, according to contents of the message, the terminal 100 may notresume reproduction of the discontinued content but may re-transmit acontrol command to the at least one external device 200.

Hereinafter, a method of reproducing, by the terminal 100, apredetermined portion of content by interoperation of the terminal 100with the at least one external device 200 will now be described indetail with reference to FIGS. 15 through 17.

FIGS. 15A and 15B illustrate screens that reproduce education contentaccording to an embodiment of the present disclosure.

Referring to FIG. 15A, the terminal 100 may reproduce the educationcontent. The terminal 100 may obtain metadata corresponding to apredetermined portion of the education content, and may identify controlinformation included in the metadata. For example, in a case in whichthe predetermined portion of the education content is a portion in whicha user directly solves a mathematical question, the metadatacorresponding to the predetermined portion may include the controlinformation for controlling a portable terminal 1500 of the user todisplay the mathematical question.

The terminal 100 may select the portable terminal 1500 as an externaldevice to reproduce the predetermined portion of the education content,and may transmit a control command to the selected portable terminal1500 so as to control the portable terminal 1500 to display themathematical question. The terminal 100 may discontinue reproduction ofthe education content until the terminal 100 receives a response messagefrom the portable terminal 1500 of the user.

Referring to FIG. 15B, the portable terminal 1500 of the user maydisplay the mathematical question on the screen according to the controlcommand from the terminal 100. In addition, the portable terminal 1500may receive an input containing a response to the mathematical questionfrom the user and may transmit the input response to the terminal 100 asthe response message with respect to the control command.

The terminal 100 may check the response message (an answer) receivedfrom the portable terminal 1500. If the answer included in the responsemessage is correct, the terminal 100 may resume reproduction of thediscontinued education content. If the answer included in the responsemessage is incorrect, the terminal 100 may transmit to the portableterminal 1500 a control command for controlling the portable terminal1500 to display a GUI on the screen so as to request another answer.

Therefore, according to various embodiments of the present disclosure,the terminal 100 may reproduce the education content by interoperatingwith the portable terminal 1500 that is the external device connected tothe terminal 100.

FIGS. 16A and 16B illustrate screens that reproduce education contentaccording to an embodiment of the present disclosure.

Referring to FIG. 16A, the terminal 100 may reproduce foreign languageeducation content. The terminal 100 may obtain metadata corresponding toa predetermined portion of the foreign language education content, andmay identify control information included in the metadata. For example,if the predetermined portion of the education content is a portion inwhich a user listens to an output voice sound and repeats the voicesound, the metadata corresponding to the predetermined portion mayinclude the control information for controlling a portable terminal 1600of the user to record a voice sound of the user.

The terminal 100 may select the portable terminal 1600 as an externaldevice to reproduce the predetermined portion of the foreign languageeducation content, and may transmit a control command to the selectedportable terminal 1600 so as to control the portable terminal 1600 torecord and transmit the voice sound of the user. The terminal 100 maydiscontinue reproduction of the foreign language education content untilthe terminal 100 receives a response message from the portable terminal1600 of the user.

Referring to FIG. 16B, the portable terminal 1600 may record the voicesound of the user according to the control command received from theterminal 100 and may transmit to the terminal 100 a file of the recordedvoice sound as a response message with respect to the control command.

The terminal 100 may check the response message (e.g., the file of therecorded voice sound) received from the portable terminal 1600. Theterminal 100 may compare the recorded voice sound included in theresponse message with the output voice sound of the terminal 100. Forexample, the terminal 100 may compare a pattern, a speed, a pitch, orthe like of the voices.

If a difference between the recorded voice sound included in theresponse message and the output voice sound of the terminal 100 iswithin a certain tolerance, the terminal 100 may resume reproduction ofthe discontinued foreign language education content. If the differencebetween the recorded voice sound included in the response message andthe output voice sound of the terminal 100 exceeds the certaintolerance, the terminal 100 may transmit to the portable terminal 1600 acontrol command for controlling the portable terminal 1600 to recordagain a voice sound of the user.

FIGS. 17A and 17B illustrate screens that reproduce cooking classcontent while a terminal interoperates with an external device accordingto another embodiment of the present disclosure.

Referring to FIG. 17A, the terminal 100 may reproduce the cooking classcontent. The terminal 100 may obtain metadata corresponding to apredetermined portion of the cooking class content and may identifycontrol information included in the metadata. For example, in a case inwhich the predetermined portion of the cooking class content is aportion in which a cook sets a temperature, time, and power of an ovenso as to use the oven, the metadata corresponding to the predeterminedportion may include the control information for controlling settingvalues of an oven 1700 of a user to be changed to values set by thecook.

The terminal 100 may select the oven 1700 as an external device toreproduce the predetermined portion of the cooking class content and maytransmit a control command to the oven 1700 to change the settingvalues. Thereafter, the terminal 100 may discontinue reproduction of thecooking class content until the terminal 100 receives a response messagefrom the oven 1700 of the user.

Referring to FIG. 17B, the oven 1700 of the user may change the settingvalues of a temperature, time, power, or the like to the values set bythe cook, according to the control command from the terminal 100.Thereafter, the oven 1700 may transmit to the terminal 100 a settingcompletion message as the response message with respect to the controlcommand.

The terminal 100 may check the response message (e.g., the settingcompletion message) received from the oven 1700. Thereafter, theterminal 100 may resume reproduction of the cooking class content. Forexample, the terminal 100 may reproduce the cooking class content fromafter the portion in which the cook sets the values of the oven.

FIG. 18 is a diagram of a content reproduction system according to anembodiment of the present disclosure.

Referring to FIG. 18, the content reproduction system may furtherinclude a gateway 300 as well as a terminal 100′ and an external device200′.

The terminal 100′ and the external device 200′ are the same as thosedescribed with reference to FIG. 1, and thus, only the gateway 300 willbe described below.

The gateway 300 may include a home gateway, a wireless relay basestation, or the like. For example, the gateway 300 may be a wired andwireless Internet router having an Internet sharing function. Inaddition, the gateway 300 may be an AP capable of performing wirelesscommunication and wired communication or a wireless router capable ofsharing an Internet service with an AP.

According to various embodiments of the present disclosure, the gateway300 may be wiredly or wirelessly connected to the terminal 100′. Inaddition, the gateway 300 may be wiredly or wirelessly connected to theexternal device 200′. For example, the gateway 300 may be connected toat least one of external device 200′-1 to 200′-N.

The gateway 300 may generate metadata about content by performing imageprocessing on the content. The gateway 300 may also manage informationabout a control protocol of the external device 200′ and may generate acontrol command that matches the control protocol of the external device200′. A method of reproducing content, the method being performed by theterminal 100′ using the gateway 300, will be described in detail withreference to FIGS. 19 and 20.

FIG. 19 is a flowchart of a method of obtaining metadata via a gatewayaccording to an embodiment of the present disclosure.

Referring to FIG. 19, at operation S1900, the terminal 100′ may transmitreproduction-target content to a metadata generating device 400 and maysimultaneously request metadata about the content. According to variousembodiments of the present disclosure, the metadata generating device400 may be the gateway 300 or may be a device for generating onlymetadata separately from the gateway 300.

At operation S1910, the metadata generating device 400 may perform imageprocessing on the content. The terminal 100′ may perform imageprocessing on the content by using various image processing algorithms(refer to the descriptions thereof in FIG. 8).

The metadata generating device 400 may recognize an atmosphere, afeeling, a device, or the like that appears in a predetermined portionof the content by performing image processing on the content.

According to various embodiments of the present disclosure, the metadatagenerating device 400 may analyze audio data included in the content.

At operation S1920, the metadata generating device 400 may generatebased on a result of the image processing a plurality of pieces ofmetadata respectively corresponding to portions of the content. Forexample, by using device information about the external device 200′related to the n^(th) portion and capability information about anoperation to be performed by the external device 200′ duringreproduction of the content which are extracted by performing imageprocessing on the n^(th) portion, the metadata generating device 400 maygenerate metadata corresponding to an n^(th) portion.

The metadata generating device 400 may insert the metadata, which isgenerated by performing image processing, directly into the content, ormay separately manage from the content the metadata as a metadata file.

At operation S1930, the metadata generating device 400 may transmit themetadata to the terminal 100′.

At operation S1940, the terminal 100′ may identify control informationby using the metadata corresponding to the predetermined portion of thecontent. For example, the terminal 100′ may identify the metadatacorresponding to the predetermined portion while the terminal reproducesthe content.

At operation S1950, the terminal 100′ may select based on the controlinformation the external device 200′ to reproduce the predeterminedportion of the content.

At operation S1960, the terminal 100′ may generate a control commandthat matches a control protocol of the external device 200′.

At operation S1970, the terminal 100′ may transmit the control commandto the external device 200. According to various embodiments of thepresent disclosure, the terminal 100 may transmit the control command tothe external device 200 via short-distance communication.

At operation S1980, the terminal 100′ may reproduce the predeterminedportion of the content by interoperating with the external device 200′.

At operation S1990, the external device 200′ may also execute thecontrol command. For example, the terminal 100′ and the external device200′ may reproduce together the predetermined portion of the content.

FIG. 20 is a flowchart of a method of converting a control command via agateway according to an embodiment of the present disclosure.

Referring to FIG. 20, at operation S2000, the terminal 100′ may bewiredly or wirelessly connected to the gateway 300.

At operation S2010, the external device 200′ may be wiredly orwirelessly connected to the gateway 300.

At operation S2020, the terminal 100′ may start reproduction of content.

At operation S2030, the terminal 100′ may obtain metadata correspondingto a predetermined portion of the content. For example, the terminal100′ may extract the metadata corresponding to the predeterminedportion, which was inserted into the content, or may generate themetadata corresponding to the predetermined portion by performing imageprocessing on the content. In addition, the terminal 100′ may requestand receive the metadata corresponding to the predetermined portion froman external server.

At operation S2040, the terminal 100′ may identify control informationincluded in the metadata corresponding to the predetermined portion. Forexample, if the predetermined portion of the content corresponds to ascene in which an ambulance passes with the siren on, the metadata mayinclude the control information of ‘device category: robot cleaner,target capability: ambulance sound output and movement, additional data:ambulance sound data, etc’.

At operation S2050, the terminal 100′ may transmit the identifiedcontrol information to the gateway 300.

At operation S2060, the gateway 300 may identify the received controlinformation and may select the external device 200′ to reproduce thepredetermined portion. Thereafter, the gateway 300 may convert thereceived control information into a control command that matches acontrol protocol of the external device 200′.

At operation S2070, the gateway 300 may transmit the control command tothe external device 200.

At operation S2080, the terminal 100 may reproduce the predeterminedportion by interoperating with the external device 200′. At operationS2090, the external device 200′ may also execute the control command.For example, the terminal 100′ and the external device 200′ mayreproduce together the predetermined portion of the content byinteroperating with each other.

FIG. 21 is a block diagram of a terminal according to an embodiment ofthe present disclosure.

Referring to FIG. 21, the terminal 100 may include a communication unit110, an output unit 120, a user input unit 130, a control unit 140 (alsoreferred as ‘processor’), and a memory 150. However, not all shownelements are necessary elements. For example, according to variousembodiments of the present disclosure, the terminal 100 may be embodiedwith more or less elements than the shown elements.

The communication unit 110 may include one or more elements for allowingcommunication between the terminal 100 and at least one external device200 or communication between the terminal 100 and the gateway 300. Forexample, the communication unit 110 may include a mobile communicationunit 111, a short-distance communication unit 112, a wired communicationunit 113, a wireless communication unit 114, and/or the like.

Examples of short-distance communication may include, but is not limitedto, Wi-Fi, Bluetooth, ZigBee, WFD, UWB, Infrared Data Association(IrDA), and BLE.

The communication unit 110 may transmit content to the gateway 300 thatconnects the terminal 100 and the at least one external device 200, andmay simultaneously request metadata corresponding to a predeterminedportion of content. Thereafter, the communication unit 110 may receivethe metadata corresponding to the predetermined portion of the contentfrom the gateway 300, wherein the metadata is generated by the gateway300 by performing image processing on the content.

The communication unit 110 may transmit a control command to the atleast one external device 200 via the gateway 300 that connects theterminal 100 and the at least one external device 200.

The output unit 120 may function to output an audio signal, a videosignal, or a vibration signal and may include a display unit 121, asound output unit 122, a vibration motor 123, or the like.

The display unit 121 displays and outputs information that is processedin the terminal 100. For example, the display unit 121 may display acontent reproduction screen, a list of external devices connected to theterminal 100, a GUI for editing metadata, or the like.

According to various embodiments of the present disclosure, the displayunit 121 may be formed as a touch screen. For example, when the displayunit 121 and a touch pad form a mutual layer structure and then areformed as a touch screen, the display unit 121 may be used as both anoutput device and input device. The display unit 121 may include atleast one of a Liquid Crystal Display (LCD), a Thin FilmTransistor-Liquid Crystal Display (TFT-LCD), an organic light-emittingdisplay device, a flexible display, a 3D display, an electrophoreticdisplay, and the like. In addition, according to a type of the terminal100, the terminal 100 may include at least two display units 121. The atleast two display units 121 may face each other by using a hinge.

The sound output unit 122 may output audio data that is received fromthe communication unit 110 or is stored in the memory 150. The soundoutput unit 122 may also output a sound signal (e.g., a call signalreceiving sound, a message receiving sound, or the like) related tocapabilities performed by the terminal 100. The sound output unit 122may include a speaker, a buzzer, or the like.

The vibration motor 123 may output a vibration signal. For example, thevibration motor 123 may output the vibration signal that corresponds toan output of the audio data (e.g., the call signal receiving sound, themessage receiving sound, or the like) or video data. In addition, when atouch is input to the touch screen, the vibration motor 123 may output avibration signal.

The user input unit 130 may be a unit by which a user inputs data so asto control the terminal 100. For example, the user input unit 130 mayinclude a key pad, a dome switch, a touch pad (a touch capacitive typetouch pad, a pressure resistive type touch pad, an infrared beamdetecting type touch pad, a surface acoustic wave type touch pad, anintegral strain gauge type touch pad, a Piezo effect type touch pad, orthe like), a jog wheel, a jog switch, and the like. However, variousembodiments of the present disclosure are not limited thereto.

The control unit 140 may generally control all operations of theterminal 100. For example, the control unit 140 may control thecommunication unit 110, the output unit 120, the user input unit 130,the memory 150, or the like by executing programs stored in the memory150.

The control unit 140 may include an application processor (not shown)and a communication processor (not shown). The application processor maycontrol execution of various applications stored in the memory 150. Thecommunication processor may control various communication functions.

The control unit 140 may establish communication with the at least oneexternal device 200 before the predetermined portion of the content isreproduced. For example, according to various embodiments of the presentdisclosure, the terminal 100 may previously scan the at least oneexternal device 200 and may therefore rapidly reproduce thepredetermined portion of the content without buffering.

The memory 150 may store a program to process and to control the controlunit 140, or may store a plurality of pieces of data (e.g., multimediacontent, device information about the at least one external device 200,an application related to the at least one external device 200,information about the control protocol of the at least one externaldevice 200, and/or the like) that are input/output.

The memory 150 may include at least one storage medium from among aflash memory-type storage medium, a hard disk-type storage medium, amultimedia card micro-type storage medium, card-type memories (e.g., anSD card, an XD memory, and the like), Random Access Memory (RAM), StaticRandom Access Memory (SRAM), Read-Only Memory (ROM), ElectricallyErasable Programmable Read-Only Memory (EEPROM), Programmable Read-OnlyMemory (PROM) magnetic memory, a magnetic disc, and an optical disc. Inaddition, the memory 150 may drive a web storage that performs a storingfunction of the memory 150 via the Internet. According to variousembodiments of the present disclosure, the memory may include anon-transitory computer-readable storage medium.

The programs stored in the memory 150 may be classified into a pluralityof modules according to functions thereof. For example, the programsstored in the memory 150 may be classified into a User Interface (UI)module 151, an external device registration module 152, a metadatamanagement module 153, a content reproduction module 154, a controlcommand conversion module 155, and/or an application module 156.However, various embodiments of the present disclosure are not limitedthereto.

The UI module 151 may provide a specialized UI or GUI whichinteroperates with the at least one external device 200 according toapplications. The UI module 151 may include a touch screen module. Thetouch screen module may detect a user's touch gesture on the touchscreen and may transmit information related to the touch gesture to thecontrol unit 140. The touch screen module may be configured as aseparate controller (hardware).

Various sensors may be arranged in or near the touch screen so as todetect a touch or a proximate touch on the touch sensor. An example ofthe sensor to detect the touch on the touch screen may be a tactilesensor. The tactile sensor detects a contact of a specific object atleast as much as a person can detect. The tactile sensor may detectvarious types of information such as the roughness of a contact surface,the hardness of the contact object, the temperature of a contact point,and/or the like.

The touch gesture of the user may include a tap gesture, a touch & holdgesture, a double tap gesture, a drag gesture, a panning gesture, aflick gesture, a drag & drop gesture, and/or the like.

“Tapping” is a user's motion of touching a screen by using a finger or atouch tool such as an electronic pen and then instantly lifting thefinger or touch tool from the screen without moving.

“Touching & holding” is a user's motion of touching a screen by using afinger or a touch tool such as an electronic pen and then maintainingthe above touching motion over a critical time (e.g., 2 seconds), aftertouching the screen. For example, a time difference between a touch-intime and a touch-out time is greater than or equal to the critical time(e.g., 2 seconds). When a touch input lasts more than the critical time,in order to remind the user whether the touch input is tapping ortouching & holding, a feedback signal may be provided in a visual,acoustic, or tactile manner. The critical time may vary according tovarious embodiments of the present disclosure. The critical time may beconfigurable according to user preferences and/or device preferences.

“Double tapping” is a user's motion of touching the screen twice byusing the finger or touch tool (e.g., such as a stylus pen).

“Dragging” is a user's motion of touching the screen by using the fingeror touch tool and moving the finger or touch tool to another position onthe screen while keeping the touching motion. The dragging motion mayenable the moving or panning motion of an object.

“Panning” is a user's motion of performing a dragging motion withoutselecting an object. Because no object is selected in the panningmotion, no object is moved in a page but the page itself is moved on thescreen or a group of objects may be moved within a page.

“Flicking” is a user's motion of performing a dragging motion over acritical speed (e.g., 100 pixel/s), by using the finger or touch tool.The dragging (panning) motion or the flicking motion may bedistinguished based on whether the moving speed of the finger or touchtool is over the critical speed (e.g., 100 pixel/s), or not. Thecritical speed may be configurable according to user preferences and/ordevice preferences.

“Dragging & Dropping” is a user's motion of dragging an object to apredetermined position on the screen by using the finger or touch tooland then dropping the object at that position.

“Pinching” is a user's motion of moving two fingers touching on thescreen in opposite directions. The pinching motion is a gesture tomagnify (open pinch) or contract (close pinch) an object or a page. Amagnification value or a contraction value is determined according tothe distance between the two fingers.

“Swiping” is a user's motion of touching an object on the screen byusing the finger or touch tool and simultaneously moving the objecthorizontally or vertically by a predetermined distance. A swiping motionin a diagonal direction may not be recognized as a swiping event.

The memory 150 may include a voice recognition module (not shown) thatrecognizes a voice of the user by using a voice recognition engine andtransmits the recognized voice to the control unit 140.

The external device registration module 152 may receive deviceinformation about the at least one external device 200 via firstshort-distance communication. According to various embodiments of thepresent disclosure, the device information about the at least oneexternal device 200 may include, but is not limited thereto, at leastone of ID information of the at least one external device 200,connection information for connecting second short-distancecommunication, a type of the at least one external device 200, acapability of the at least one external device 200, information about acategory and a protocol used to control the at least one external device200, information about an application for controlling the at least oneexternal device 200, and the like.

When an application related to the at least one external device 200 isnot installed in the terminal 100, the external device registrationmodule 152 may download and install the application related to the atleast one external device 200.

The external device registration module 152 may store and manage thedevice information about the at least one external device 200 in anexternal device DataBase (DB). When the device information about the atleast one external device 200 is changed, the external deviceregistration module 152 may update the device information stored in theexternal device DB.

The metadata management module 153 may obtain metadata corresponding tothe predetermined portion of the content. For example, the metadatamanagement module 153 may extract the metadata inserted into thecontent, may receive the metadata corresponding to the predeterminedportion of the content from an external source, or may generate themetadata corresponding to the predetermined portion of the content byperforming image processing on the content. The metadata managementmodule 153 may generate the metadata corresponding to the predeterminedportion of the content by referring to a type of an external deviceconnected to the terminal 100. In addition, when the metadata managementmodule 153 detects connection of another external device duringreproduction of the content, the metadata management module 153 maygenerate the metadata corresponding to the predetermined portion byfurther referring to the connected other external device.

When reproduction of the content is complete or the content is deleted,the metadata management module 153 may delete the metadata so as toefficiently manage the memory 150.

The content reproduction module 154 may reproduce the content inresponse to a user request. The content reproduction module 154 mayreproduce the predetermined portion of the content by interoperatingwith the at least one external device 200.

According to various embodiments of the present disclosure, the contentmay include, but is not limited to, education content, movie content,broadcasting content, game content, and the like.

When the control command is transmitted to the at least one externaldevice 200, the content reproduction module 154 may discontinuereproduction of the content, and when a message corresponding to thecontrol command is received from the at least one external device 200,the content reproduction module 154 may resume reproduction of thediscontinued content.

The content reproduction module 154 may identify the control informationincluded in the metadata corresponding to the predetermined portion ofthe content and may transmit the control information to the controlcommand conversion module 155.

The control command conversion module 155 may select the at least oneexternal device 200 that is to reproduce the predetermined portion ofthe content and that is from among one or more external devicesconnected to the terminal 100, based on the control information.Thereafter, the control command conversion module 155 may convert thecontrol information into the control command that matches the controlprotocol of the at least one external device 200.

According to various embodiments of the present disclosure, the controlcommand conversion module 155 may be included in each of applicationsfor controlling one or more external devices 200, respectively. Forexample, the respective applications for controlling the respectiveexternal devices 200 may have information about respective controlprotocols of the respective external devices 200, and may generaterespective control commands to be read by the respective externaldevices 200, by using the information about the respective controlprotocols.

The application module 156 may be connected to an application DB and theexternal device DB. The application DB may store one or moreapplications for controlling one or more external devices 200,respectively. For example, the application DB may store an applicationfor controlling a medical apparatus, an application for controlling asporting equipment, an application for controlling a home gateway, anapplication for controlling a CE device, or the like. The externaldevice DB may store one or more pieces of device information about theone or more external devices 200.

The application module 156 may control the at least one external device200 by using an application for controlling the at least one externaldevice 200. For example, the application module 156 may transmit acontrol command for controlling the at least one external device 200 tothe at least one external device 200.

One or more embodiments of the present disclosure may also be embodiedas programmed commands to be executed in various computer means, andthen may be recorded to a non-transitory computer-readable recordingmedium. The non-transitory computer-readable recording medium mayinclude one or more of the programmed commands, data files, datastructures, or the like. The programmed commands recorded on thenon-transitory computer-readable recording medium may be particularlydesigned or configured for one or more embodiments of the presentdisclosure or may be well known to one of ordinary skill in the art.Examples of the non-transitory computer-readable recording mediuminclude magnetic media including hard disks, magnetic tapes, and floppydisks, optical media including CD-ROMs and DVDs, magneto-optical mediaincluding optical disks, and a hardware apparatus designed to store andexecute the programmed commands in ROM, RAM, a flash memory, and thelike. Examples of the programmed commands include not only machine codesgenerated by a compiler but also codes to be executed in a computer byusing an interpreter. The hardware apparatus may be configured tofunction as one or more software modules so as to perform operations ofone or more embodiments of the present disclosure, or vice versa.

While the present disclosure has been shown and described with referenceto various embodiments thereof, it will be understood by those skilledin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present disclosure asdefined by the appended claims and their equivalents.

What is claimed is:
 1. A content reproduction apparatus, the apparatuscomprising: a display; a communication interface; and at least oneprocessor configured to: control the display to display image data of acontent which is being reproduced on the display, control thecommunication interface to establish a connection between the contentreproduction apparatus and at least one external device, extract acharacteristic of the content by analyzing audio data of the content,obtain, from the extracted characteristic, information about anoperation to be performed by the at least one external device toreproduce an audio portion of the content, and control the at least oneexternal device to perform the operation of reproducing the audioportion of the content based on the information obtained.
 2. Theapparatus of claim 1, wherein the content reproduction apparatus and theat least one external device are synchronized such that the contentreproduction apparatus displays the image data of the content and the atleast one external device performs the operation of reproducing theaudio portion of the content.
 3. The apparatus of claim 1, wherein theat least one processor is further configured to select, from among aplurality of external devices connected to the content reproductionapparatus, the at least one external device that is to reproduce aportion of the content.
 4. The apparatus of claim 1, wherein the atleast one processor is further configured to control the display todisplay a list of a plurality of external devices connected to thecontent reproduction apparatus.
 5. The apparatus of claim 1, wherein theat least one processor is further configured to: execute a firstapplication for reproducing the content, and execute a secondapplication for controlling the at least one external device to performthe operation of reproducing the audio portion of the content.
 6. Theapparatus of claim 5, wherein the first application transmits theinformation obtained from the extracted characteristic to the secondapplication, and wherein the second application converts the informationinto a control command according to a control protocol of the at leastone external device.
 7. The apparatus of claim 1, wherein the at leastone processor is further configured to: perform image processing on thecontent, and obtain, from the result of the image processing of thecontent, additional information about the operation to be performed bythe at least one external device to reproduce the audio portion of thecontent.
 8. The apparatus of claim 1, wherein the at least one processoris further configured to control the communication interface to:transmit the content to a gateway that connects the content reproductionapparatus to the at least one external device, contemporaneously requestthe information about the operation to be performed by the at least oneexternal device, and receive information about the operation to beperformed by the at least one external device to reproduce the audioportion of the content from the gateway.
 9. The apparatus of claim 1,wherein the at least one processor is further configured to control thecommunication interface to transmit a control command to the at leastone external device via a gateway that connects the content reproductionapparatus to the at least one external device.
 10. A method ofreproducing content by a content reproduction apparatus, the methodcomprising: displaying image data of a content which is being reproducedon a display of the content reproduction apparatus; extracting acharacteristic of the content by analyzing audio data of the content;obtaining, from the extracted characteristic, information about anoperation to be performed by at least one external device to reproducean audio portion of the content; and controlling, by the contentreproduction apparatus, the at least one external device to perform theoperation of reproducing the audio portion of the content based on theinformation obtained.
 11. The method of claim 10, wherein the contentreproduction apparatus and the at least one external device aresynchronized such that the content reproduction apparatus displays theimage data of the content and the at least one external device performsthe operation of reproducing the audio portion of the content.
 12. Themethod of claim 10, further comprising: selecting, from among aplurality of external devices connected to the content reproductionapparatus, the at least one external device that is to reproduce aportion of the content.
 13. The method of claim 10, further comprising:displaying a list of a plurality of external devices connected to thecontent reproduction apparatus.
 14. The method of claim 10, furthercomprising: executing a first application for reproducing the contentand a second application for controlling the at least one externaldevice to perform the operation of reproducing the audio portion of thecontent.
 15. The method of claim 14, wherein the first applicationtransmits the information obtained from the extracted characteristic tothe second application, and wherein the second application converts theinformation into a control command according to a control protocol ofthe at least one external device.
 16. The method of claim 10, whereinthe obtaining of the information further comprises: performing imageprocessing on the content, and obtaining, from the result of the imageprocessing of the content, additional information about the operation tobe performed by the at least one external device to reproduce the audioportion of the content.
 17. The method of claim 10, wherein theobtaining of the information comprises: transmitting the content to agateway that connects the content reproduction apparatus to the at leastone external device, contemporaneously requesting the information aboutthe operation to be performed by the at least one external device, andreceiving information about the operation to be performed by the atleast one external device to reproduce the audio portion of the contentfrom the gateway.
 18. A non-transitory computer-readable recordingmedium having recorded thereon one or more computer programs which, whenexecuted by at least one processor, cause the at least one processor tocontrol for: displaying image data of a content which is beingreproduced on a display of a content reproduction apparatus; extractinga characteristic of the content by analyzing audio data of the content;obtaining, from the extracted characteristic, information about anoperation to be performed by at least one external device to reproducean audio portion of the content; and controlling, by the contentreproduction apparatus, the at least one external device to perform theoperation of reproducing the audio portion of the content based on theinformation obtained.